Server device, conference assistance system, and conference assistance method

ABSTRACT

Provided is a server device with which a participant in a conference can easily acquire information pertaining to another participant. The server device is provided with an acquisition unit and an information provision unit. The acquisition unit acquires a profile of each of a plurality of users using a conference assistance system. The information provision unit provides, to a first participant participating in a conference from among a plurality of users, a profile pertaining to a second participant participating in the same conference as that in which the first participant is participating.

TECHNICAL FIELD

The present invention relates to a server device, a conferenceassistance system, a conference assistance method, and a program.

BACKGROUND ART

In recent years, with advances of network technologies and informationprocessing technologies, video conferences in which conference rooms atremote places are connected via networks have become active.

For example, PTL 1 describes that content of a conference is capitalizedto make a conference operation efficient. A conference assistance systemdisclosed in PTL 1 includes an image recognition unit. The imagerecognition unit recognizes an image related to each participant fromvideo data acquired by a video conference device using an imagerecognition technology. Further, the system includes a voice recognitionunit. The voice recognition unit acquires voice data of each attendeeacquired by the video conference device and compares the voice data withfeature information of the voice of each attendee registered in advance.Further, the voice recognition unit identifies a speaker of each speechact in the voice data on the basis of information regarding movement ofeach attendee. Further, the conference assistance system includes atimeline management unit that outputs voice data of each attendeeacquired by the voice recognition unit in time-series speeches as atimeline.

CITATION LIST Patent Literature

[PTL 1] JP 2019-061594 A

SUMMARY OF INVENTION Technical Problem

As described above, so-called video conferences are actively performed.In a normal conference, when participants who meet for the first timeattends the conference, the participants exchange their business cards.The participants can obtain information regarding people from theacquired business cards.

However, business cards cannot be exchanged in a video conferenceperformed by connecting remote places via a network, and theparticipants cannot obtain information regarding people whom they meetfor the first time. Although a simple self-introduction or the like maybe performed at the beginning of the conference, the participants onlyknow the names or the like of the relevant people. However, in theconference, the positions of other participants, particularly, thepositions and the like of business partners are important informationfor proceeding with the conference. Not only in video conferences butalso at in-house meetings, there are many companies and the like inwhich business cards are not exchanged. Even in such cases, a problemmay arise in that it is difficult to obtain information necessary for aconference.

A main objective of the present invention is to provide a server device,a conference assistance system, a conference assistance method, and aprogram contributing to easy acquisition of information regarding otherparticipants of a conference by each participant.

Solution to Problem

According to a first aspect of the present invention, there is provideda server device including: an acquisition unit configured to acquire aprofile of each of a plurality of users using a conference assistancesystem; and an information provision unit configured to provide a firstparticipant participating in a conference among the plurality of userswith a profile related to a second participant participating in the sameconference as the conference in which the first participantparticipates.

According to a second aspect of the present invention, there is provideda conference assistance system including: a first terminal used by afirst participant participating in a conference; a second terminal usedby a second participant participating in the same conference as theconference in which the first participant participates; and a serverdevice. The server device includes an acquisition unit that acquires aprofile of each of a plurality of users who uses the system and aninformation provision unit that provides a profile related to the secondparticipant to the first terminal.

According to a third aspect of the present invention, there is provideda conference assistance method including: in a server device, acquiringa profile of each of a plurality of users using a conference assistancesystem; and providing a first participant participating in a conferenceamong the plurality of users with a profile related to a secondparticipant participating in the same conference as the conference inwhich the first participant participates.

According to a fourth aspect of the present invention, there is provideda computer-readable recording medium that stores a program causing acomputer mounted on a server device to execute: a process of acquiring aprofile of each of a plurality of users using a conference assistancesystem; and a process of providing a first participant participating ina conference among the plurality of users with a profile related to asecond participant participating in the same conference as theconference in which the first participant participates.

Advantageous Effects of Invention

According to each aspect of the present invention, there is provided aserver device, a conference assistance system, a conference assistancemethod, and a program contributing to easy acquisition of informationregarding other participants of a conference by each participant. Theadvantageous effects of the present invention are not limited to theabove advantageous effects. According to the present invention, othereffects may be exhibited instead of or in addition to the advantageouseffects.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating an overview of an example embodiment.

FIG. 2 is a diagram illustrating an example of an overall configurationof a conference assistance system according to a first exampleembodiment.

FIG. 3 is a diagram illustrating an example of a screen displayed on aconference room terminal.

FIG. 4 is a diagram illustrating an example of a processingconfiguration of a server device according to the first exampleembodiment.

FIG. 5 is a diagram illustrating an example of a processingconfiguration of a user registration unit according to the first exampleembodiment.

FIG. 6 is a diagram illustrating an operation of a user informationacquisition unit according to the first example embodiment.

FIG. 7 is a diagram illustrating an example of a user database accordingto the first example embodiment.

FIG. 8 is a diagram illustrating an example of a participant listaccording to the first example embodiment.

FIG. 9 is a diagram illustrating an example of a processingconfiguration of the conference room terminal according to the firstexample embodiment.

FIG. 10 is a diagram illustrating an operation of a profile informationrequest unit according to the first example embodiment.

FIG. 11 is a diagram illustrating the operation of a profile informationoutput unit according to the first example embodiment.

FIG. 12 is a sequence diagram illustrating an example of an operation ofthe conference assistance system according to the first exampleembodiment.

FIG. 13 is a diagram illustrating an operation of a user informationacquisition unit according to a second example embodiment.

FIG. 14 is a diagram illustrating an example of a user databaseaccording to the second example embodiment.

FIG. 15 is a diagram illustrating an operation of a conference roomterminal according to a third example embodiment.

FIG. 16 is a diagram illustrating an operation of the conference roomterminal according to the third example embodiment.

FIG. 17 is a diagram illustrating an example of a processingconfiguration of the conference room terminal according to the thirdexample embodiment.

FIG. 18 is a diagram illustrating an example of a processingconfiguration of a server device according to the third exampleembodiment.

FIG. 19 is a diagram illustrating an example of a processingconfiguration of a speaker information transmission unit according tothe third example embodiment.

FIG. 20 is a diagram illustrating an example of speaker informationaccording to the third example embodiment.

FIG. 21 is a diagram illustrating an operation of a participant’sopinion management unit according to the third example embodiment.

FIG. 22 is a diagram illustrating an example of a participant’s viewdatabase according to the third example embodiment.

FIG. 23 is a diagram illustrating an operation of the conference roomterminal according to the third example embodiment.

FIG. 24 is a diagram illustrating an operation of the conference roomterminal according to the third example embodiment.

FIG. 25 is a sequence diagram illustrating an example of an operation ofa conference assistance system according to the third exampleembodiment.

FIG. 26 is a diagram illustrating an example of a hardware configurationof a server device.

FIG. 27 is a diagram illustrating an operation of a conference roomterminal according to a modified example of the present disclosure.

EXAMPLE EMBODIMENT

First, an overview of an example embodiment will be described. Thereference numerals in the drawings attached to this overview areattached to each element for convenience as an example for assistingunderstanding, and the description of the overview is not intended to belimiting in any way. Unless otherwise particularly stated, the blocksdescribed in each drawing represent not a configuration of a hardwareunit but a configuration of a functional unit. Connection lines betweenblocks in each drawing include both bidirectional and unidirectionallines. A unidirectional arrow schematically indicates a flow of a mainsignal (data) and does not exclude bidirectionality. In the presentspecification and the drawings, elements that can be similarly describedare denoted by the same reference numerals, and redundant descriptionmay be omitted.

A server device 100 according to an example embodiment includes anacquisition unit 101 and an information provision unit 102 (see FIG. 1). The acquisition unit 101 acquires a profile of each of a plurality ofusers using a conference assistance system. The information provisionunit 102 provides a first participant participating in a conference fromamong the plurality of users with a profile of a second participantparticipating in the same conference as the conference in which thefirst participant participates.

The server device 100 acquires the profile of the user in advance andregisters the profile, for example, in a database. For example, whenprovision of information regarding other participants is requested froma participant, the server device 100 reads necessary information fromthe database and provides the information to the participant making therequest to provide the information. As a result, even when theparticipants meet for the first time, the participants can easilyacquire information regarding the other participants.

Hereinafter, specific example embodiments will be described in moredetail with reference to the drawings.

First Example Embodiment

A first example embodiment will be described in more detail withreference to the drawings.

FIG. 2 is a diagram illustrating an example of an overall configurationof a conference assistance system according to the first exampleembodiment. As illustrated in FIG. 2 , a plurality of bases is connectedand a video conference is performed. A conference room terminal isinstalled in a conference room of each base. In the example of FIG. 2 ,conference room terminals 10-1 to 10-4 are installed in the conferenceroom of the base A. Similarly, conference room terminals 10-5 to 10-8are installed in the conference room of the base B. In the followingdescription, when there is no particular reason to distinguish theconference room terminals 10-1 to 10-8 from each other, they are simplyreferred to as “conference room terminals 10.”

Although not illustrated in FIG. 2 , in the conference room of eachbase, a camera device that can acquire a bird’s eye view image of theentire conference room and a speaker that outputs voice of the partnerare installed.

Each of the plurality of conference room terminals 10 and the serverdevice 20 are connected by wired or wireless communication means and areable to communicate with each other. The server device 20 may beinstalled in either the base A or the base B or may be installed on anetwork (on a cloud) as illustrated in FIG. 2 .

The configuration illustrated in FIG. 2 is exemplary and is, of course,not intended to limit the number of bases participating in the videoconference or the number of conference room terminals 10 or the like. Inthe following example embodiments, a conference assistance system thatassists with a video conference connecting a plurality of bases will bemainly described. However, the conference to which the conferenceassistance system according to the present disclosure is applied is notlimited to a video conference, and a conference held in the sameconference room may be an assistance target.

The conference room terminal 10 is a terminal installed in each seat ofthe conference room. The participant manipulates the terminal to performthe conference while displaying necessary information or the like. Theconference room terminal 10 has a camera function and is able to image aparticipant who is seated. Further, the conference room terminal 10 canbe connected to a microphone (for example, a pin microphone or awireless microphone). A voice of a participant seated in front of eachof the conference room terminals 10 is collected by the microphone. Themicrophone connected to the conference room terminal 10 is preferably amicrophone with strong directivity. This is because it is necessary tocollect a voice of a user wearing the microphone, and it is notnecessary to collect voices of other people.

The server device 20 is a device that assists with a conference. Theserver device 20 assists with a conference which is a place for decisionmaking and a place for idea thinking. The server device 20 enables avideo conference performed in a plurality of bases. The server device 20transmits data (packets) related to a voice and a video acquired fromone base to other bases. For example, in the example of FIG. 2 , voiceand video data acquired from the base A are transmitted to the base B.

The server device 20 transmits bird’s eye view video data of one base tothe other bases. The video data may be displayed on the conference roomterminal 10 used by each participant or may be projected by a projectoror the like. The server device 20 transmits voice data acquired from onebase (voice data acquired from the conference room terminals 10) to theother bases. The voice data may be output from the speaker installed inthe conference room or may be output from each conference room terminal10.

In this way, the server device 20 enables an existing video conferencesystem. An existing technology can be used to enable a video conferencesystem by the server device 20. Therefore, in the following description,implementation and installation of the video conference system will beomitted.

The server device 20 enables of the video conference system and also hasa function of providing detailed information regarding participants(attribute information of the participants). For example, when there arefour participants in a video conference, as illustrated in FIG. 2 , theserver device 20 displays information regarding the three otherparticipants on the conference room terminal 10 manipulated by eachparticipant. The server device 20 assists with a conference (videoconference) by providing information regarding the other participants.In the following description, detailed information regarding theparticipant (attribute information such as a face image and a name) isreferred to as “participant profile information” or “profileinformation.”

For example, a screen as illustrated in FIG. 3 is displayed on theconference room terminal 10-1 used by the participant U1 illustrated inFIG. 2 . Referring to FIG. 3 , a screen overlooking the base B (aconference room of the base B) is displayed in an upper portion (a mainregion) of the display area. Profile information of the otherparticipants is displayed in a lower portion of the display area (asub-region: a region surrounded by a dotted line). In the example ofFIG. 3 , the profile information of the participants who participate inthe video conference in the same base and the profile information of theparticipants who participate in the video conference in the other baseare displayed.

The participant manipulates the conference room terminal 10 in front ofthe participant and inputs an instruction indicating that she or hewants to view the participant profile information to the server device20. The conference room terminal 10 receiving the manipulation transmitsa “profile provision request” to the server device 20. As a response tothe request, the server device 20 transmits data (information) enablingdisplay illustrated in FIG. 3 to the conference room terminal 10.

Of course, the display screen illustrated in FIG. 3 is exemplary and isnot intended to limit content, a layout, and the like to be displayed.For example, the profile information of the participant participating inthe video conference from the same base may not be displayed, and themain region in the upper portion of the display area may not bedisplayed when the profile information is displayed. The contentdisplayed as the profile of the participant is not limited to FIG. 3 ,and other information (for example, a workplace or the like) may bedisplayed.

Preliminary Preparation

Here, to enable conference assistance with the server device 20, asystem user (a user scheduled to participate in a video conference) isrequired to make a preliminary preparation. The preliminary preparationwill be described below.

The user registers attribute values such as her or his own biometricinformation (a face image) and her or his own profile in the system.Specifically, the user inputs her or his face image to the server device20. The user inputs her or his profile (for example, information such asa name, an employee number, a workplace, an affiliation department, aposition, and a contact) to the server device 20.

Any method can be used to input information such as the biometricinformation and the profile. For example, the user captures her or hisface image using a terminal such as a smartphone. Further, the usergenerates a text file or the like in which the profile is describedusing the terminal. The user manipulates the terminal to transmit theinformation (the face image and the profile) to the server device 20.Alternatively, the user may input necessary information to the serverdevice 20 using an external storage device such as a Universal SerialBus (USB) in which the information is stored.

Alternatively, the server device 20 may have a function of a web serverand the user may input necessary information in a form provided by theweb server. Alternatively, a terminal inputting the information may beinstalled in each conference room and the user may input necessaryinformation from the terminal installed in the conference room to theserver device 20.

The server device 20 updates a database that manages the system userusing the acquired user information (the biometric information, theprofiles, and the like). Details of the updating of the database will bedescribed below. The server device 20 generally updates the database inaccordance with the following operation. In the following description, adatabase managing users using the system according to the presentdisclosure will be referred to as a “user database.”

When a person relevant to the acquired user information is a new userwho is not registered in the user database, the server device 20 assignsan identifier (ID) to the user. The server device 20 generates a featureamount that characterizes the acquired face image.

The server device 20 adds an entry including the ID assigned to the newuser, the feature amount generated from the face image, the face imageof the user, the profile, and the like to the user database. When theserver device 20 registers the user information, the participants in theconference can use the conference assistance system illustrated in FIG.2 .

Next, details of each device included in the conference assistancesystem according to the first example embodiment will be described.

Server Device

FIG. 4 is a diagram illustrating an example of a processingconfiguration (processing module) of the server device 20 according tothe first example embodiment. Referring to FIG. 4 , the server device 20includes a communication control unit 201, a user registration unit 202,a participant identification unit 203, a profile information provisionunit 204, and a storage unit 205.

The communication control unit 201 is means configured to controlcommunication with other devices. Specifically, the communicationcontrol unit 201 receives data (packets) from the conference roomterminal 10. The communication control unit 201 transmits data to theconference room terminal 10. The communication control unit 201 deliversdata received from other devices to other processing modules. Thecommunication control unit 201 transmits data acquired from otherprocessing modules to other devices. In this way, the other processingmodules transmit and receive data to and from other devices via thecommunication control unit 201.

The user registration unit 202 is means configured to enable theabove-described registration of the system user. The user registrationunit 202 acquires at least a profile of each of a plurality of usersusing the conference assistance system. The user registration unit 202includes a plurality of submodules. FIG. 5 is a diagram illustrating anexample of a processing configuration of the user registration unit 202.Referring to FIG. 5 , the user registration unit 202 includes a userinformation acquisition unit 211, an ID generation unit 212, a featureamount generation unit 213, and an entry management unit 214.

The user information acquisition unit 211 is means configured to acquirethe above-described user information. The user information acquisitionunit 211 acquires biometric information (a face image) and a profile (aname, affiliation, and the like) of each system user. The system usermay input the information from a terminal such as a smartphone to theserver device 20 or may directly manipulate the server device 20 toinput the information.

The user information acquisition unit 211 may provide a graphical userinterface (GUI) or a form for inputting the information. For example,the user information acquisition unit 211 displays an information inputform illustrated in FIG. 6 on the terminal manipulated by the user.

The system user inputs the information illustrated in FIG. 6 . Inaddition, the system user selects whether to newly register a user inthe system or to update the already registered information. When all theinformation is input, the system user presses a “Transmit” button andinputs the biometric information, the profile, and the like to theserver device 20.

The user information acquisition unit 211 stores the acquired userinformation in the storage unit 205.

The ID generation unit 212 is means configured to generate an ID to beassigned to the system user. The ID generation unit 212 generates a userID for identifying each of the plurality of system users. Morespecifically, when the user information input by the system user isinformation regarding new registration, the ID generation unit 212generates an ID for identifying a new user. For example, the IDgeneration unit 212 may calculate a hash value of the acquired userinformation (the face image and the profile) and use the hash value asan ID to be assigned to the user. Alternatively, the ID generation unit212 may assign a unique value and use the assigned value as the IDwhenever a user is registered. In the following description, an IDgenerated by the ID generation unit 212 (an ID for identifying a systemuser) is referred to as a “user ID.”

The feature amount generation unit 213 is means configured to generate afeature amount (a feature vector including a plurality of featureamounts) characterizing a face image from the face image included in theuser information. Specifically, the feature amount generation unit 213extracts feature points from the acquired face image. An existingtechnology can be used for the feature point extraction process, andthus detailed description thereof will be omitted. For example, thefeature amount generation unit 213 extracts eyes, a nose, a mouth, andthe like as feature points from the face image. Thereafter, the featureamount generation unit 213 calculates a position of each feature pointor a distance between the feature points as a feature amount andgenerates a feature vector including a plurality of feature amounts.

The entry management unit 214 is means configured to manage an entry ofthe user database. When a new user is registered in the database, theentry management unit 214 adds an entry including the generated user ID,the generated feature amounts, the face image acquired from the terminalor the like of the user, and the profile to the user database.

When information regarding the user already registered in the userdatabase is updated, the entry management unit 214 identifies an entryto be subjected to information updating based on an employee number orthe like, and updates the user database using the acquired userinformation. At that time, the entry management unit 214 may update adifference between the acquired user information and the informationregistered in the database or may overwrite each item of the databasewith the acquired user information. Similarly, with regard to thefeature amount, the entry management unit 214 may update the databasewhen there is a difference in the generated feature amount or mayoverwrite the existing feature amount with the newly generated featureamount.

The user registration unit 202 constructs, for example, a user databaseas illustrated in FIG. 7 . It goes without saying that the contentregistered in the user database illustrated in FIG. 7 is exemplary andis not intended to limit the information registered in the userdatabase. For example, the “feature amount” may not be registered in theuser database as necessary. That is, when there is a face image, it ispossible to calculate the feature amount and thus the feature amount maybe calculated as necessary.

The description will now return to FIG. 4 . The participantidentification unit 203 is means configured to identify a participantparticipating in the conference (a participant in the video conferenceamong users registered in the system). The participant identificationunit 203 acquires a face image from the conference room terminal 10 inwhich the participant is seated among the conference room terminals 10installed in the conference room. The participant identification unit203 calculates a feature amount from the acquired face image.

The participant identification unit 203 sets a feature amount calculatedbased on the face image acquired from the conference room terminal 10 asa comparison target and performs a comparison process with the featureamount registered in the user database. More specifically, theparticipant identification unit 203 sets the above-calculated featureamount (feature vector) as a comparison target and performs one-to-N(where N is a positive integer, the same applies below) comparisons witha plurality of feature vectors registered in the user database.

The participant identification unit 203 calculates similarity between acomparison target feature amount and each of the plurality of featureamounts of a registration side. In the similarity, a chi-squaredistance, a Euclidean distance, or the like can be used. The longer thedistance is, the lower the similarity is. The shorter the distance is,the higher similarity is.

The participant identification unit 203 identifies a feature amount withsimilarity with the feature amount of the comparison target equal to orgreater than a predetermined value and having the highest similarityamong the plurality of feature amounts registered in the user database.

The participant identification unit 203 reads the user ID relevant tothe feature amount obtained as results of the one-to-N comparisons fromthe user database.

The participant identification unit 203 repeats the above-describedprocess on the face image acquired from each of the conference roomterminals 10, and identifies the user ID relevant to each face image.The participant identification unit 203 generates a participant list byassociating the identified user ID, the ID of the conference roomterminal 10 which is a transmission source of the face image, and thebase of the participant. As the ID of the conference room terminal 10, amedia access control (MAC) address or an Internet protocol (IP) addressof the conference room terminal 10 can be used.

For example, in the example of FIG. 2 , a participant list illustratedin FIG. 8 is generated. In FIG. 8 , to facilitate understanding,reference numerals assigned to the conference room terminal 10 aredescribed as conference room terminal IDs. A “participant ID” includedin the participant list is a user ID registered in the user database.

Note that, by inputting a relationship between the conference roomterminal ID and the base (conference room) where each conference roomterminal 10 is installed to the server device 20 in advance, the serverdevice 20 can ascertain from which base each participant participates inthe video conference using the conference room terminal ID.

The profile information provision unit 204 is means configured toprovide profile information regarding other participants to theparticipants of the video conference. The profile information provisionunit 204 provides one participant participating in a conference fromamong a plurality of system users with a profile regarding the otherparticipants participating in the same conference as the conference inwhich the one participant participates.

The profile information provision unit 204 processes the “profileprovision request” acquired from the conference room terminal 10.Specifically, the profile information provision unit 204 identifies aconference room terminal ID of a transmission source of the request. Theprofile information provision unit 204 identifies the participant makingthe request from the identified conference room terminal ID withreference to the participant list.

The profile information provision unit 204 identifies a participant IDthat is a participant ID included in the participant list other than theparticipant transmitting the profile provision request. For example, inthe example of FIG. 8 , when the profile provision request istransmitted from the conference room terminal 10-1, the participant IDsof the IDs 02 to 04 are identified.

The profile information provision unit 204 extracts (collects)information to be transmitted as participant profile information fromeach field of the entry relevant to the identified participant ID withreference to the user database. For example, as illustrated in FIG. 3 ,when “face image,” “name,” “affiliation,” and “position” of theparticipant are displayed as profile information, the profileinformation provision unit 204 reads the information from the userdatabase.

Alternatively, the profile information provision unit 204 may readregistration values (attribute values such as names) of all the fieldsregistered in the user database from the database in such a manner thatinformation can be selected in the conference room terminal 10.

The profile information provision unit 204 transmits a responseincluding the read information (a response to the profile provisionrequest) to the conference room terminal 10 which is a transmissionsource of the request.

The storage unit 205 is means configured to store information necessaryfor the operation of the server device 20.

Conference Room Terminal

FIG. 9 is a diagram illustrating an example of a processingconfiguration (a processing module) of the conference room terminal 10.Referring to FIG. 9 , the conference room terminal 10 includes acommunication control unit 301, a face image acquisition unit 302, avoice transmission unit 303, a profile information request unit 304, aprofile information output unit 305, and a storage unit 306.

The communication control unit 301 is means configured to controlcommunication with other devices. Specifically, the communicationcontrol unit 301 receives data (packets) from the server device 20. Thecommunication control unit 301 transmits data to the server device 20.The communication control unit 301 delivers data received from otherdevices to other processing modules. The communication control unit 301transmits data acquired from other processing modules to other devices.In this way, the other processing modules transmit and receive data toand from other devices via the communication control unit 301.

The face image acquisition unit 302 is means configured to control thecamera device and acquire a face image (biometric information) of aparticipant seated in front of the own device. The face imageacquisition unit 302 images the front of the own device periodically orat a predetermined timing. The face image acquisition unit 302determines whether a face image of a person is included in the acquiredimage and extracts the face image from the acquired image data when theface image is included. The face image acquisition unit 302 transmits aset of the extracted face image and the ID (a conference room terminalID; for example, an IP address) of the own device to the server device20.

Since an existing technology can be used for a face image detectionprocess and a face image extraction process by the face imageacquisition unit 302, detailed description thereof will be omitted. Forexample, the face image acquisition unit 302 may extract a face image (aface area) from image data by using a learning model learned with aconvolutional neural network (CNN). Alternatively, the face imageacquisition unit 302 may extract a face image using a scheme such astemplate matching.

The voice transmission unit 303 is means configured to acquire a voiceof a participant and transmit the acquired voice to the server device20. The voice transmission unit 303 acquires a voice file related to avoice collected by a microphone (for example, a pin microphone). Forexample, the voice transmission unit 303 acquires a voice file encodedin a format such as a waveform audio file (WAV file).

The voice transmission unit 303 analyzes the acquired voice file andtransmits a voice file including a voice section (a section in whichthere is no silence; a speech of a participant) to the server device 20when the voice file includes the voice section. At that time, the voicetransmission unit 303 transmits the voice file and the ID (theconference room terminal ID) of the own device to the server device 20.

Alternatively, the voice transmission unit 303 may assign the conferenceroom terminal ID to the voice file acquired from the microphone andtransmit the voice file, as it is, to the server device 20. In thiscase, the voice file acquired by the server device 20 may be analyzed toextract the voice file including the voice.

The voice transmission unit 303 extracts a voice file (a voice file withno silence) including a speech of a participant by using an existing“voice detection technology.” For example, the voice transmission unit303 detects the voice using a voice parameter sequence or the likemodeled by a hidden Markov model (HMM).

The profile information request unit 304 is means configured to generatethe above-described “profile provision request” in response to amanipulation of the participant and transmit the request to the serverdevice 20. For example, the profile information request unit 304generates a GUI used for the participant to input information. Forexample, the profile information request unit 304 displays a screen asillustrated in FIG. 10 on the display.

When the participant desires to provide information regarding profilesof other people, the profile information request unit 304 transmits aprofile provision request including the ID (the conference room terminalID) of the own device to the server device 20.

The profile information request unit 304 acquires a response to therequest from the server device 20. The profile information request unit304 delivers the acquired response to the profile information outputunit 305.

The profile information output unit 305 is means configured to outputparticipant profile information regarding other participants based onthe response acquired from the server device 20. For example, theprofile information output unit 305 performs display illustrated in FIG.3 using the information acquired from the server device 20.

Alternatively, the profile information output unit 305 may performdisplay illustrated in FIG. 11 . Referring to FIG. 11 , the profileinformation output unit 305 simply displays each participant in thesub-region indicated by the dotted line. The profile information outputunit 305 may display more detailed information regarding a personselected by the participant (a person surrounded by an alternate longand short dash line in FIG. 11 ) in the main area. In this case, theserver device 20 transmits more detailed information regarding eachparticipant to the conference room terminal 10.

Alternatively, the profile information output unit 305 may include abutton or the like for switching an item to be displayed and changecontent to be displayed in response to a press of the button.

Alternatively, the profile information output unit 305 may display theprofile on a face photo of a participant. For example, the profileinformation output unit 305 may perform display such as augmentedreality (AR) on the face photo with regard to the profile.

Alternatively, the profile information output unit 305 may emphasize aprofile of a participant who is speaking (for example, the profile ofthe participant is enlarged and surrounded by a dotted line or thelike.). Alternatively, the profile information output unit 305 mayhighlight and display the profile according to attributes of otherparticipants (for example, when the participant is a person outside thecompany, a director, or the like). Alternatively, the profileinformation output unit 305 may display a linked profile regarding acontact of another participant. The participant may make a contact theother person using the link. Alternatively, the profile informationoutput unit 305 may select a profile and display a “registration” buttonor the like for registering the selected profile in the contact book.

The storage unit 306 is means configured to store information necessaryfor an operation of the conference room terminal 10.

Operation of Conference Assistance System

Next, an operation of the conference assistance system according to thefirst example embodiment will be described.

FIG. 12 is a sequence diagram illustrating an example of an operation ofthe conference assistance system according to the first exampleembodiment. FIG. 12 is a sequence diagram illustrating an example of asystem operation when a conference is actually held. It is assumed thata system user is registered in advance before the operation of FIG. 12 .

When a conference starts and participants are seated, the conferenceroom terminal 10 acquires face images of the seated participants andtransmits the face images to the server device 20 (step S01).

The server device 20 identifies the participants using the acquired faceimages (step S11). The server device 20 identifies the participants ofthe conference among a plurality of system users registered in advancethrough a comparison process using the face images acquired from theparticipants and the face images registered in the user database. Morespecifically, the server device 20 sets feature amounts calculated fromthe acquired face images as feature amounts of a comparison side andsets a plurality of feature amounts registered in the user database asfeature amounts of the registration side and executes one-to-Ncomparison. The server device 20 repeats the comparison for eachparticipant (the conference room terminal 10 used by the participant) inthe conference and generates a participant list.

A participant who intends to know the profiles or the like of otherparticipants during the conference inputs the intention to theconference room terminal 10. The conference room terminal 10 transmits a“profile provision request” to the server device 20 in response to theinput (step S02).

The server device 20 acquires the face images, the profiles, and thelike of the participants participating in the conference with referenceto the user database. Thereafter, the server device 20 transmits aresponse including the acquired face images, the profiles, and the liketo the conference room terminal 10 (step S12).

The conference room terminal 10 outputs the participant profileinformation based on the acquired response (step S03).

As described above, the server device 20 according to the first exampleembodiment includes a user database that stores a user ID generated bythe ID generation unit 212, a face image of a user, and a profile inassociation. The profile information provision unit 204 of the serverdevice 20 acquires the face image and the profile of the participantsfrom the user database using the user ID of the participantparticipating in the conference. The server device 20 transmits theacquired face image and profile to the conference room terminal 10 usedby the participant. As a result, even when the participants meet eachother for the first time, the participants can easily acquireinformation regarding the other participants by manipulating theconference room terminal 10 in front and transmitting the profileprovision requests of the other participants to the server device 20.

Second Example Embodiment

Next, a second example embodiment will be described in detail withreference to the drawings.

In the second example embodiment, a case in which a plurality ofattribute values (setting values) is permitted for some or all of theitems among items input as profiles by users will be described. Forexample, the second example embodiment will be described on theassumption that a system user work as an employee of a company and aprofessor of a university. In the second example embodiment, a case inwhich an optimum attribute value is automatically selected from theplurality of attribute values when the participant profile informationis displayed on the conference room terminal 10 will be described.

The configuration of the conference assistance system according to thesecond example embodiment can be the same as that of the first exampleembodiment, and thus description equivalent to FIG. 2 will be omitted.Since processing configurations of the conference room terminal 10 andthe server device 20 according to the second example embodiment can bethe same as those of the first example embodiment, description thereofwill be omitted. Hereinafter, differences between the first and secondexample embodiments will be mainly described. The first participant usesa first terminal (for example, the conference room terminal 10-1), andthe second participant uses a second terminal (for example, theconference room terminal 10-5).

The user registration unit 202 of the server device 20 can acquire aplurality of attribute values for the same item with regard to theprofiles of the system users. For example, the user informationacquisition unit 211 of the user registration unit 202 displays aninformation input form illustrated in FIG. 13 on the terminalmanipulated by the user.

For example, when the system users belong to different organizations(companies, universities, or the like), the system users inputinformation regarding each affiliation organization. When all theinformation is input, the system user presses a “Transmit” button andinputs the biometric information, the profile, and the like to theserver device 20.

The user information acquisition unit 211 stores the acquired userinformation in the storage unit 205.

The entry management unit 214 of the user registration unit 202 adds anentry including the profile acquired from the user to the user database.As a result, the user database as illustrated in FIG. 14 is constructed.

When a profile provision request is received, the profile informationprovision unit 204 generates a response to the profile provisionrequest. At this time, when a plurality of attribute values is set forthe same item (the same type of item) among items of a profileregistered in the user database, the profile information provision unit204 selects an optimum attribute value from the plurality of attributevalues and includes the selected attribute value in the response to therequest.

For example, in the example of FIG. 14 , for a user with ID01, twoattribute values are set for each of the affiliation organization andthe affiliation department. In this case, the profile informationprovision unit 204 selects one of the affiliation organizations 1 and 2and one of the affiliation departments 1 and 2 when the profileinformation regarding the user with ID01 is generated.

The profile information provision unit 204 selects an optimum attributevalue from a plurality of attribute values set to the same item on thebasis of various standards and rules.

For example, the profile information provision unit 204 may select anoptimum attribute value on the basis of a day of week on which theconference is held.

For example, the profile information provision unit 204 may select anoptimum attribute value according to a purpose of the conference. Forexample, when the profile provision request is made, the participantinputs the purpose of the conference to the server device 20. Forexample, the conference room terminal 10 may display an item forinputting “purpose of conference” on the GUI illustrated in FIG. 10 . Amanager or the like inputs (registers) a relationship between thepurpose of the conference and the attribute value to be selected to theserver device 20 in advance as table information.

For example, the profile information provision unit 204 may select anoptimum attribute value from a conversation between the participants.The profile information provision unit 204 analyzes a voice acquiredfrom the conference room terminal 10 and determines a purpose of theconversation. Specifically, the profile information provision unit 204extracts a keyword from the speech of the participant and calculates aspeech frequency of the extracted keyword. The profile informationprovision unit 204 determines that the keyword with a high speechfrequency clearly indicates the purpose of the conversation anddetermines the purpose of the conference.

The manager or the like inputs the keyword extracted by the profileinformation provision unit 204 to the server device 20 in advance. Themanager or the like also inputs table information in which a keywordwith a high speech frequency is associated with the purpose of theconference or table information in which the purpose of the conferenceis associated with an attribute value to be selected to the serverdevice 20 in advance. When the keyword is extracted from the speech ofthe participant, it is necessary to transform the speech (voice) intotext, but the description of the transformation into text can be madewith reference to description of the text transformation unit 222 to bedescribed below.

For example, the profile information provision unit 204 may determine anattribute value to be selected from the profile of the participant. Forexample, the profile information provision unit 204 may select the sameattribute value as the attribute value of an information provisionrequester. For example, in the example of FIG. 14 , the affiliationorganizations of user IDs 01, 03, and 04 are the same. When the usersparticipate in the same conference and acquire an information provisionrequest from the participant with ID03, the profile informationprovision unit 204 selects the same affiliation organization as theparticipant with ID03 when the affiliation organization related to theparticipant with ID01 is selected.

In the second example embodiment, the server device 20 selects anoptimum attribute value among a plurality of attribute values related tothe same item. Further, the server device 20 presents the selectedattribute value to the participant in the conference room terminal 10,and thus convenience is improved. However, the conference room terminal10 may display a plurality of attribute values. For example, the profileinformation provision unit 204 may assign priority to each of aplurality of attribute values and transmit the attribute values to theconference room terminal 10. For example, the profile informationprovision unit 204 sets the priority of the attribute value selected bythe above-described method to be high. The server device 20 transmits aresponse including the profile including the attribute value to whichthe priority is assigned to the conference room terminal 10. Theconference room terminal 10 may display the plurality of acquiredattribute values and display the emphasized attribute values with highpriority.

In the above description, the case in which the server device 20automatically selects one or more attribute values from the plurality ofattribute values of the same item has been described, but the automaticselection may be performed in the conference room terminal 10. In thiscase, the server device 20 may transmit attribute values of all thefields of the participants of the conference registered in the userdatabase to the conference room terminal 10.

When one attribute value is selected from the plurality of attributevalues and one attribute value is selected from the plurality ofattribute values of other items, the server device 20 may associate theselection of the attribute value. For example, when “university” isselected as the attribute value of the workplace, a mail address of“university” is selected as the attribute value of the contact inassociation.

As described above, when the profile of the user is acquired, the serverdevice 20 according to the second example embodiment can acquire aplurality of attribute values (for example, a company, a university, orthe like as a workplace) for the same item (for example, an affiliationorganization or the like). When there are items of which a plurality ofattribute values is set in the profile of the other participants, theserver device 20 selects one or more attribute values among theplurality of attribute values and provides the selected one or moreattribute values to the participants. That is, the server device 20selects an optimum attribute value among the plurality of attributevalues related to the same item and provides the optimum attribute valueto the participant. Therefore, the participant can recognize theaffiliation or the like of other participants accurately and can have amore appropriate discussion.

Third Example Embodiment

Next, a third example embodiment will be described in detail withreference to the drawings.

In the third example embodiment, in addition to display of profileinformation of a participant, a case in which a simple opinionexpression (intention expression) for a speech of another participantcan be performed will be described.

Since a configuration of the conference assistance system according tothe third example embodiment can be the same as those of the first andsecond example embodiments, description equivalent to FIG. 2 is omitted.Hereinafter, differences between the first to third example embodimentswill be mainly described.

For example, as illustrated in FIG. 15 , the conference room terminal 10displays a face photo of the participant and her or his speech. At thattime, when a participant agrees with a speech of another participant,the conference room terminal 10 provides an interface through which theparticipant inputs the agreement of the speech to the conference roomterminal 10. For example, the conference room terminal 10 displays an“agreement button” illustrated in FIG. 15 and acquires an opinion of aparticipant about a speech of another participant by pressing thebutton.

The server device 20 generates statistical information of an opinionregarding the speech of another participant for each participant,includes the generated statistical information in a response to the“profile provision request,” and transmits the generated statisticalinformation to the conference room terminal 10. In the followingdescription, statistical information regarding the opinion of theparticipant with respect to the speech of another person is referred toas “participant’s opinion statistical information”. For example, theserver device 20 generates a ratio (agreement ratio) of speeches withwhich each participant has agreed to the speeches of the participant U1(speeches in the entire conference) as a participant’s opinionstatistical information.

The conference room terminal 10 performs display illustrated in FIG. 16using the received response. The participant in contact with the displayillustrated in FIG. 16 can know opinions (views) of the otherparticipants regarding the own speech. In a normal video conference, itis difficult to know the thinking of other participants on a screen.This is because it is difficult for the participant to perceive a subtlechange in an expression of a partner as in a face-to-face conference.However, with the conference assistance by the server device 30according to the third example embodiment, the participants can know theopinions of other participants about the own speeches.

Conference Room Terminal

FIG. 17 is a diagram illustrating an example of a processingconfiguration (processing module) of the conference room terminal 10according to the third example embodiment. Referring to FIG. 17 , anopinion acquisition unit 307 is added to the configuration of theconference room terminal 10 according to the first example embodiment.

The opinion acquisition unit 307 is means configured to acquire anopinion of a participant about a speech of another person (an opinion ofa participant who operates the own device). The opinion acquisition unit307 acquires “speaker information” from the server device 20. As detailsof the speaker information will be described below, the informationincludes an ID (a speech ID to be described below) with which a speechis identified, a face photo of a current speaker, speech content, andthe name of the speaker.

The opinion acquisition unit 307 generates an interface used for theparticipant to express her or his opinion on the displayed speech whiledisplaying the information included in the speaker information. Forexample, the opinion acquisition unit 307 displays an “agreement button”as illustrated in FIG. 15 .

The display illustrated in FIG. 15 is exemplary, and various forms andforms can be considered in the display by the opinion acquisition unit307. For example, the opinion acquisition unit 307 may display a buttonindicating “disagreement” with respect to the speech. Alternatively, theopinion acquisition unit 307 may display a “neutral button” indicatingthat a speech is neither agreed nor disagreed.

The opinion acquisition unit 307 transmits thinking of the participant(a view of the participant) with respect to the speech of anotherparticipant to the server device 20. Specifically, the opinionacquisition unit 307 transmits, to the server device 20, the speech IDof the speech in which the participant has expressed the opinion, theconference room terminal ID of the own device, and the opinion of theparticipant (for example, agreement, disagreement, and neutrality of aspeech). In the following description, information transmitted from theopinion acquisition unit 307 to the server device 20 is referred to as“participant’s view information.” That is, the information for notifyingthe server device 20 of her or his own thinking and view of the speechof another participant is the “participant’s view information.”

The profile information output unit 305 displays the profile informationon the basis of a response received from the server device. At thattime, the profile information output unit 305 displays statisticalinformation (participants’ opinions statistical information; forexample, an agreement ratio for the speech) regarding the opinions ofother participants about the speech of the participant seated in frontof the own device. For example, the profile information output unit 305performs display as illustrated in FIG. 16 .

Server Device

FIG. 18 is a diagram illustrating an example of a processingconfiguration (processing module) of the server device 20 according tothe third example embodiment. Referring to FIG. 18 , a speakerinformation transmission unit 206 and a participant’s opinion managementunit 207 are added to the configuration of the server device 20according to the first example embodiment.

The speaker information transmission unit 206 is means configured togenerate information (speaker information) regarding a speech when thespeech of the participant is detected, and transmit the information tothe conference room terminal 10. When the speech of the participant isdetected, the speaker information transmission unit 206 transmitsspeaker information including at least speech content by the participantto the conference room terminal 10 (a terminal other than the conferenceroom terminal 10 used by the speaker).

The speaker information transmission unit 206 includes a plurality ofsubmodules. FIG. 19 is a diagram illustrating an example of a processingconfiguration of the speaker information transmission unit 206.Referring to FIG. 19 , the speaker information transmission unit 206includes a voice acquisition unit 221, a text transformation unit 222,and an information transmission unit 223.

The voice acquisition unit 221 is means configured to acquire a voice ofa participant from the conference room terminal 10. The conference roomterminal 10 generates a voice file of each speech of the participant andtransmits the voice file to the server device 20 along with an ID of theown device (a conference room terminal ID). The voice acquisition unit221 identifies a participant ID relevant to the acquired conference roomterminal ID with reference to the participant list. The voiceacquisition unit 221 delivers the identified participant ID and thevoice file acquired from the conference room terminal 10 to the texttransformation unit 222.

The text transformation unit 222 is means configured to transform theacquired voice file into text. The text transformation unit 222transforms content recorded in a voice file into text using a voicerecognition technology. Since the text transformation unit 222 can usean existing voice recognition technology, detailed description thereofis omitted. The text transformation unit 222 operates as follows.

The text transformation unit 222 performs filtering process of removingnoise or the like from the voice file. Next, the text transformationunit 222 identifies a phoneme from a sound wave of the voice file. Thephoneme is the smallest unit of a language. The text transformation unit222 identifies a sequence of phonemes and transforms the sequence into aword. The text transformation unit 222 generates a sentence from asequence of words and outputs a text file. In the filtering process, avoice smaller than a predetermined level is deleted. Therefore, evenwhen a voice of the neighbor is contained in the voice file, the textfile is not generated from the voice of the neighbor.

The text transformation unit 222 delivers the participant ID and thetext file to the information transmission unit 223.

The information transmission unit 223 is means configured to transmitspeaker information to the conference room terminal 10. The informationtransmission unit 223 acquires a name, a face image, and the likerelevant to the participant ID acquired from the text transformationunit 222 with reference to the user database.

The information transmission unit 223 generates “speaker information”including the name of the speaker, the face image of the speaker, andthe speech content (a text file of the speech content) and transmits thegenerated speaker information to the conference room terminal 10. Atthat time, the information transmission unit 223 includes an ID foridentifying speaker information (hereinafter referred to as a speech ID)in the speaker information. For example, the information transmissionunit 223 transmits the speaker information illustrated in FIG. 20 toeach conference room terminal 10. Any value can be used as the speech IDas long as the value is information with the speech can be uniquelyidentified. For example, the information transmission unit 223 cangenerate the speech ID by increasing the value of the speech ID wheneverthe speaker information is generated.

In addition, the information transmission unit 223 delivers theparticipant ID, the speech ID, and the speech content (the text file) ofthe speaker to the participant’s opinion management unit 207.

The participant’s opinion management unit 207 is means configured tomanage information (participant’s view information) indicating anopinion of a participant about a speech of another person. Morespecifically, the participant’s opinion management unit 207 manages adatabase storing each participant’s view about the speech of anotherperson (hereinafter referred to as a participant’s view database).

To generate and manage the participant’s view database, theparticipant’s opinion management unit 207 generates a conference recordon the basis of the participant ID, the speech ID, and the speechcontent acquired from the information transmission unit 223.Specifically, the participant’s opinion management unit 207 stores aspeech time, the participant ID, the speech ID, and the speech contentin association. As a result, a conference record as illustrated in FIG.21 is generated. In FIG. 21 , the participant ID of the speaker isreferred to as a “speaker ID.”

The participant’s opinion management unit 207 acquires “participant’sview information” from each conference room terminal 10. Theparticipant’s opinion management unit 207 reflects the obtainedparticipant’s view information in the conference record. For example,when a participant’s opinion is “agreement,” the participant’s opinionmanagement unit 207 adds the agreement of the participant about thespeech of another person to the conference record.

The participant’s opinion management unit 207 identifies a speech ofwhich the participant has expressed an opinion on the basis of thespeech ID (a speech ID included in the participant’s view informationfrom among speeches included in the conference record). Theparticipant’s opinion management unit 207 generates, for example, tableinformation (participant’s view database) illustrated in FIG. 22 byreflecting participant’s opinions in the identified speech. Theparticipant’s view database is a database that manages the speechcontent by the participants and views of other participants inassociation. In FIG. 22 , when each participant expresses an opinion of“agreement” about the speech, a flag is set (“1” is set).

At the time of a response to the profile provision request acquired fromthe conference room terminal 10, the profile information provision unit204 includes statistical information regarding an opinion (view) ofanother person about the speech of the participant who has transmittedthe information request. Specifically, the profile information provisionunit 204 generates “participant’s opinion statistical information” withreference to the participant’s view database. As described above, theparticipant’s opinion statistical information is statistical informationregarding an opinion of a participant about a speech of another person.

For example, the profile information provision unit 204 generates, asparticipant’s opinion statistical information, a ratio (an agreementratio) of speeches that each participant agrees with speeches of othersin the entire conference. That is, the profile information provisionunit 204 calculates, as the agreement ratio, a ratio of speeches withwhich other participants agree among speeches by the participants.

The profile information provision unit 204 calculates a total number ofspeeches of one participant (a participant manipulating the conferenceroom terminal 10 which is a transmission source of a profile informationprovision request) with reference to the participant’s view database.Specifically, the profile information provision unit 204 calculates atotal number of speaker IDs (participant IDs) relevant to the oneparticipant among entries of the participant’s view database. Further,the profile information provision unit 204 counts the number of entriesin which a flag is set among the entries of the speaker ID relevant tothe one participant with respect to the participants other than the oneparticipant. The profile information provision unit 204 calculates anagreement ratio of each participant by calculating the ratio of theentry in which the flag is set to the calculated total number ofentries.

In the example of FIG. 22 , agreement ratios of other participants to aspeech of the participant with speaker ID01 are participant ID02=50%,participant ID03=100%, and participant ID04=0%. The result indicatesthat an opinion of participant ID03 is affirmative (positive) and anopinion of participant ID04 is oppose (negative) with respect to aspeech of the participant ID01 of the participant.

The profile information provision unit 204 includes the generatedparticipant’s opinion statistical information in a response to theprofile provision request and transmits the response to the conferenceroom terminal 10 which is a transmission source of the request.

Note that the profile information provision unit 204 may generateanother index as the participant’s opinion statistical informationinstead of the agreement ratio. For example, a “disagreement ratio” tospeeches of other people may be generated as participant’s opinionstatistical information.

Alternatively, the profile information provision unit 204 may generatethe “agreement ratio” or the “disagreement ratio” for each time zone asparticipant’s opinion statistical information. For example, the profileinformation provision unit 204 may calculate an agreement ratio for eachpredetermined period (for example, every 10 minutes) and transmit theagreement ratio as participant’s opinion statistical information to theconference room terminal 10. In this case, the conference room terminal10 may display a temporal change of an agreement ratio or the like (seeFIG. 23 ).

Alternatively, the profile information provision unit 204 may generateparticipant’s opinion statistical information of a base different fromthat of the participant of the information provision request. Theprofile information provision unit 204 identifies a transmission sourceof the profile information provision from the conference room terminalID. The profile information provision unit 204 identifies a participantID of the base different from that of the identified conference roomterminal ID with reference to the participant list. The profileinformation provision unit 204 calculates an agreement ratio to theidentified participant ID of the other base with reference to theparticipant’s view database. For example, the profile informationprovision unit 204 calculates the agreement ratio for each participantID of the other base and calculates an average value of the calculatedagreement ratios to calculate the agreement ratio on the other base.

For example, in the example of FIG. 2 , when the participant U1 (theconference room terminal 10-1 manipulated by the participant U1)transmits the profile information provision request, the profileinformation provision unit 204 generates participant’s opinionstatistical information of the entire base B. For example, in theexample of FIG. 22 , the participant IDs (speaker IDs) of theparticipants U1 and U2 in the base A are ID01 and ID02, respectively. Inaddition, participant IDs (speaker IDs) of participants U3 and U4 in thebase B are ID03 and ID04, respectively. The agreement ratio of the baseB to the speeches of the participant U1 of the base A is calculated as50% (2/4). In this case, the conference room terminal 10 may display anagreement ratio of the entire base of the partner side (a base agreementratio) in a main region in which a bird’s eye view of the base of thepartner side is displayed (see FIG. 24 ). For the base agreement ratio,an agreement ratio for each time zone may be calculated.

Operation of Conference Assistance System

Next, an operation of the conference assistance system according to thethird example embodiment will be described.

FIG. 25 is a sequence diagram illustrating an example of the operationof the conference assistance system according to the third exampleembodiment. FIG. 25 is a sequence diagram illustrating an example of asystem operation when a conference is actually held and mainlyillustrates differences from the first example embodiment.

When the participant makes a speech during the conference, theconference room terminal 10 transmits voices of the participants to theserver device 20 (step S21).

The server device 20 acquires the voices and generates “speakerinformation” including the speech content. The server device 20transmits the generated speaker information to the conference roomterminal 10 (step S31).

The conference room terminal 10 displays a screen for inputting anopinion of a participant (a view of a participant about the speech)using the acquired speaker information. Specifically, the conferenceroom terminal 10 provides the participant with an interface asillustrated in FIG. 15 . The conference room terminal 10 acquires viewsof the participants on the interface and transmits “participant’s viewinformation” including the content (for example, agreement,disagreement, neutrality, or the like) to the server device 20 (stepS22).

The server device 20 updates the participant’s view database using theacquired participation view information (step S32).

During the conference, the operations of steps S21 to S32 are repeated,and the views of the participants are collected in the participant’sview database.

During the conference, the conference room terminal 10 transmits aprofile provision request to the server device 20 (step S23).

The server device 20 generates the participant’s opinion statisticalinformation at a timing at which the profile provision request isreceived (step S33).

The server device 20 transmits a response (a response to the profileprovision information) including the generated participant’s opinionstatistical information to the conference room terminal 10 (step S34).

The conference room terminal 10 outputs the participant profileinformation based on the acquired response (step S24).

As described above, the server device according to the third exampleembodiment collects the opinions expressed by the participants of theconference about the speeches of other participants and provides theparticipants with information regarding the opinions of other peoplewhen there are requests from by the participants. As a result, even in avideo conference or the like in which it is difficult to ascertain anexpression change of the other people, the participants can know thethinking of the other participants by confirming the “agreement ratio”or the like displayed on the conference room terminal 10.

Next, hardware of each device included in the conference assistancesystem will be described. FIG. 26 is a diagram illustrating an exampleof a hardware configuration of the server device 20.

The server device 20 can be configured with an information processingdevice (a so-called computer) and has the configuration illustrated inFIG. 26 . For example, the server device 20 includes a processor 311, amemory 312, an input/output interface 313, and a communication interface314. The constituent elements such as the processor 311 are connectedvia an internal bus or the like to be able to communicate with eachother.

However, the configuration illustrated in FIG. 26 is not intended tolimit the hardware configuration of the server device 20. The serverdevice 20 may include hardware (not illustrated) or may not include theinput/output interface 313 as necessary. The number of processors 311and the like included in the server device 20 is not limited to theexample of FIG. 26 . For example, the plurality of processors 311 may beincluded in the server device 20.

The processor 311 is, for example, a programmable device such as acentral processing unit (CPU), a micro processing unit (MPU), or adigital signal processor (DSP). Alternatively, the processor 311 may bea device such as a field programmable gate array (FPGA) or anapplication specific integrated circuit (ASIC). The processor 311executes various programs including an operating system (OS).

The memory 312 is a random access memory (RAM), a read-only memory(ROM), a hard disk drive (HDD), a solid state drive (SSD), or the like.The memory 312 stores an OS program, an application program, and variouskinds of data.

The input/output interface 313 is an interface of a display device or aninput device (not illustrated). The display device is, for example, aliquid crystal display or the like. The input device is, for example, adevice such as a keyboard or a mouse that receives a user manipulation.

The communication interface 314 is a circuit, a module, or the like thatcommunicates with other devices. For example, the communicationinterface 314 includes a network interface card (NIC).

Functions of the server device 20 are enabled by various processingmodules. The processing modules are enabled, for example, by theprocessor 311 executing programs stored in the memory 312. The programscan be recorded on a computer-readable recording medium. The recordingmedium may be a non-transitory medium such as a semiconductor memory, ahard disk, a magnetic recording medium, or an optical recording medium.That is, the present invention can also be enabled as a computer programproduct. The program can be downloaded via a network or updated using arecording medium that stores the program. Further, the processingmodules may be enabled by a semiconductor chip.

The conference room terminal 10 can also be configured with aninformation processing device similarly to the server device 20. Sincethere is no difference in the basic hardware configuration from theserver device 20, description thereof will be omitted. The conferenceroom terminal 10 may include a camera and a microphone, or may beconnectable with a camera and a microphone.

The server device 20 is equipped with a computer and the functions ofthe server device 20 are enabled by causing the computer to executeprograms. The server device 20 executes a conference assistance methodin accordance with the program.

Modified Examples

The configurations, the operations, and the like of the conferenceassistance system described in the above example embodiment are merelyexemplary and are not intended to limit the configuration and the likeof the system.

In the above example embodiment, a microphone is connected to theconference room terminal 10 and a speaker is identified with an ID ofthe conference room terminal 10 that transmits a voice. However, onemicrophone may be installed at a desk in each base and the microphonemay collect a speech of each participant. In this case, the serverdevice 20 may perform “speaker identification” on the voice collectedfrom the microphone to identify the speaker.

In the above example embodiment, the case in which the dedicatedconference room terminal 10 is installed on the desk has been described,but the function of the conference room terminal 10 may be enabled by aterminal held (possessed) by the participant. For example, each of theparticipants may participate in the conference using their ownterminals. The participant operates the own terminal and transmits theown face image to the server device 20 when a conference starts. Theterminal transmits the voice of the participant to the server device 20.The server device 20 may provide an image, a video, or the like to theparticipant using a projector.

The profile of the system user (an attribute value of the user) may beinput using a scanner or the like. For example, the user inputs an imagerelated to the own business card to the server device 20 using ascanner. The server device 20 performs optical character recognition(OCR) processing on the acquired images. The server device 20 maydetermine the profile of the user based on the obtained information.

In the above example embodiment, the case in which biometric informationrelated to a “face image” is transmitted from the conference roomterminal 10 to the server device 20 has been described. However, thebiometric information related to “a feature amount generated from theface image” may be transmitted from the conference room terminal 10 tothe server device 20. The server device 20 may perform a process ofcomparing a feature amount registered in the user database with anacquired feature amount (a feature vector).

In the above example embodiment, when a request (a profile informationprovision request) is received from the conference room terminal 10, theserver device 20 transmits profile information of other participants tothe conference room terminal 10. However, the server device 20 maytransmit profile information of other participants to each of theconference room terminals 10 at the beginning of the conference. Forexample, the server device 20 may transmit the profile information ofeach participant to the conference room terminal 10 at a timing at whichthe identification of all the participants in the conference ends (atiming at which generation of a participant list ends).

When the server device 20 retains layout information (map information)of each base, positional information of a person who is seated may betransmitted to the conference room terminal 10 using the layoutinformation. For example, when the server device 20 retains layoutinformation of two bases illustrated in FIG. 2 , the positionalinformation of the participants is identified with a conference roomterminal ID. The server device 20 transmits the layout informationincluding two bases and positional information of the participants U1 toU4 to the conference room terminal 10. The conference room terminal 10acquiring the positional information generates an interface, asillustrated in FIG. 27 . The conference room terminal 10 may displayprofile information of a person designated by the interface illustratedin FIG. 27 .

In the above example embodiment, the case in which the conference roomterminal 10 displays the profile information or the like when theprofile information of the other participant is acquired has beendescribed. The conference room terminal 10 may use the acquired profileinformation for other purposes. For example, the conference roomterminal 10 may group the other participants on the basis of anagreement ratio acquired from the server device 20. For example, theconference room terminal 10 may display the participants separately in agroup with a high agreement ratio and a group having a low agreementratio. Alternatively, the conference room terminal 10 may generate arelationship map between the participants using the foregoing grouping.

The server device 20 may transmit the number of times each participantspeaks to the conference room terminal 10. The conference room terminal10 may visualize an amount of speech. Alternatively, the conference roomterminal 10 may perform a display or the like of encouraging theparticipants to speak more actively to a participant whose speech amountis less than a predetermined threshold value.

The server device 20 may notify the conference room terminal 10 of avoice volume (magnitude of voice) of each participant. The conferenceroom terminal 10 performs various kinds of control according to thenotified voice volume. For example, the conference room terminal 10performs a display for encouraging a participant with a small voice toraise her or his volume. Alternatively, the conference room terminal 10takes measures such as an increase in an amplification factor (gain) ofa microphone that collects a voice of a person with a small voice.

The server device 20 may analyze a speech of each participant,automatically extract points at issue of the conference, and transmitthe extracted points at issue to the conference room terminal 10. Forexample, the server device 20 sets keywords (predetermined keywords)that frequently appear in speeches of each participant as points atissue of the conference and transmits the keywords to the conferenceroom terminal 10. The conference room terminal 10 may display the pointsat issue, provide an interface acquiring an intention (the pros and consof the points at issue) of the participant and acquire a view of theparticipant. The server device 20 may collect (aggregate) the views ofthe respective participants and provide the aggregation result to theconference room terminal 10. The aggregation result may be displayed onthe conference room terminal 10.

In the flow diagram (a flowchart and a sequence diagram) used in theabove description, the plurality of steps (processes) has been describedin order, but the execution order of the steps executed in the exampleembodiment is not limited to the described order. In the exampleembodiment, for example, the order of the illustrated steps can bechanged within a range in which there is no problem in terms of content,such as executing the processes in parallel.

To facilitate understanding of the present disclosure, the above exampleembodiments have been described in detail and it is not intended thatall the above-described configurations are necessary. When a pluralityof example embodiments has been described, each example embodiment maybe used alone or in combination. For example, a part of a configurationof an example embodiment can be replaced with a configuration of anotherexample embodiment or a configuration of another example embodiment canbe added to a configuration of an example embodiment. Further, otherconfigurations can be added, deleted, and replaced in a part of aconfiguration of an example embodiment.

Although the industrial applicability of the present invention isapparent from the above description, the present invention can beappropriately applied to a system or the like that assists with aconference or the like held by a company or the like.

The whole or part of the example embodiments disclosed above can bedescribed as, but not limited to, the following supplementary notes.

Supplementary Note 1

A server device comprising:

-   an acquisition unit configured to acquire a profile of each of a    plurality of users using a conference assistance system; and-   an information provision unit configured to provide a first    participant participating in a conference among the plurality of    users with a profile related to a second participant participating    in the same conference as the conference in which the first    participant participates.

Supplementary Note 2

The server device according to supplementary note 1,

-   in which the acquisition unit acquires a face image and a profile of    each of the plurality of users, and-   the information provision unit provides the first participant with a    face image and a profile of the second participant.

Supplementary Note 3

The server device according to supplementary note 2,

-   in which the acquisition unit generates a user ID for identifying    each of the plurality of users,-   the server device further comprises a user database configured to    store the generated user IDs, face images, and profiles in    association, and-   the information provision unit acquires the face image and the    profile of the second participant from the user database using a    user ID of the second participant.

Supplementary Note 4

The server device according to supplementary note 3, further comprisinga participant identification unit that identifies the first and secondparticipants among the plurality of users through a comparison processusing the face images acquired from the first and second participantsand the face images registered in the user database.

Supplementary Note 5

A conference assistance system comprising:

-   a first terminal used by a first participant participating in a    conference;-   a second terminal used by a second participant participating in the    same conference as the conference in which the first participant    participates; and-   a server device,-   in which the server device includes-   an acquisition unit that acquires a profile of each of a plurality    of users who uses the system, and-   an information provision unit that provides a profile related to the    second participant to the first terminal.

Supplementary Note 6

The conference assistance system according to supplementary note 5,

-   in which the acquisition unit acquires a face image and the profile    of each of the plurality of users, and-   the information provision unit provides a face image and a profile    of the second participant to the first terminal.

Supplementary Note 7

The conference assistance system according to supplementary note 6,

-   in which the acquisition unit generates a user ID for identifying    each of the plurality of users,-   the server device further includes a user database that stores the    generated user ID, face image, and profile in association, and-   the information provision unit acquires the face image and the    profile of the second participant from the user database by using a    user ID of the second participant.

Supplementary Note 8

The conference assistance system according to supplementary note 7,

-   in which the server device further includes-   a participant identification unit that identifies the first and    second participants among the plurality of users through a    comparison process using the face images acquired from the first and    second participants and the face images registered in the user    database.

Supplementary Note 9

The conference assistance system according to any one of supplementarynotes 5 to 8, in which the first terminal requests the server device toprovide information regarding the profile related to the secondparticipant.

Supplementary Note 10

The conference assistance system according to supplementary note 9, inwhich the first terminal acquires a response to the request forinformation provision of the profile from the server device and displaysthe profile related to the second participant based on the acquiredresponse.

Supplementary Note 11

The conference assistance system according to supplementary note 10, inwhich the server device further includes:

-   a speaker information transmission unit that transmits, to the    second terminal, speaker information including at least content of a    speech by the first participant when the speech of the first    participant is detected,-   the second terminal transmits view information including a view of    the second participant on the content of the speech by the first    participant to the server device,-   the server device further includes-   a participant’s view database that manages the content of the speech    by the first participant and a view of the second participant in    association, and-   the information provision unit transmits a response including    statistical information regarding the view of the second participant    to the first terminal when a request to provide information    regarding the profile is acquired from the first terminal.

Supplementary Note 12

The conference assistance system according to supplementary note 11, inwhich the information provision unit calculates a ratio of speech withwhich the second participant agrees among speech by the firstparticipant as an agreement ratio and transmits a response including thecalculated agreement ratio to the first terminal.

Supplementary Note 13

The conference assistance system according to supplementary note 11 or12, in which the second terminal transmits agreement or disagreement ofthe second participant about the content of the speeches of the firstparticipant as the view information to the server device.

Supplementary Note 14

A conference assistance method comprising: in a server device, acquiringa profile of each of a plurality of users using a conference assistancesystem; and

-   providing a first participant participating in a conference among    the plurality of users with a profile related to a second    participant participating in the same conference as the conference    in which the first participant participates.

Supplementary Note 15

A computer-readable recording medium that stores a program causing acomputer mounted on a server device to execute:

-   a process of acquiring a profile of each of a plurality of users    using a conference assistance system; and-   a process of providing a first participant participating in a    conference among the plurality of users with a profile related to a    second participant participating in the same conference as the    conference in which the first participant participates.

The disclosures of the cited literatures are incorporated herein byreference. Although the exemplary embodiments of the present inventionhave been described above, the present invention is not limited to theexemplary embodiments. It will be understood by those of ordinary skillin the art that these embodiments are exemplary and that variousmodifications are possible without departing from the scope and spiritof the present invention. That is, it goes without saying that thepresent invention includes various modifications and corrections thatcan be made by those of ordinary skill in the art in accordance with theentire disclosure including the claims and the technical idea.

REFERENCE SIGNS LIST

-   10, 10-1 to 10-8 Conference room terminal-   20, 100 Server device-   101 Acquisition unit-   102 Information provision unit-   201, 301 Communication control unit-   202 User registration unit-   203 Participant identification unit-   204 Profile information provision unit-   205, 306 Storage unit-   206 Speaker information transmission unit-   207 Participant’s opinion management unit-   211 User information acquisition unit-   212 ID generation unit-   213 Feature amount generation unit-   214 Entry management unit-   221 Voice acquisition unit-   222 Text transformation unit-   223 Information transmission unit-   302 Face image acquisition unit-   303 Voice transmission unit-   304 Profile information request unit-   305 Profile information output unit-   307 Opinion acquisition unit-   311 Processor-   312 Memory-   313 Input/output interface-   314 Communication interface

What is claimed is:
 1. A server device comprising: one or more memoriesstoring instructions; and one or more processors configured to executethe instructions to: acquire a profile of each of a plurality of usersusing a conference assistance system; and provide a first participantparticipating in a conference among the plurality of users with aprofile related to a second participant participating in the sameconference as the conference in which the first participantparticipates.
 2. The server device according to claim 1, wherein the oneor more processors are configured to execute the instructions to:acquire a face image and a profile of each of the plurality of users,and provide the first participant with a face image and a profile of thesecond participant.
 3. The server device according to claim 2, whereinthe one or more processors are configured to execute the instructionsto: generate a user ID for identifying each of the plurality of users,and acquire the face image and the profile of the second participantfrom a user database that stores the generated user IDs, face images,and profiles in association using a user ID of the second participant.4. The server device according to claim 3, wherein the one or moreprocessors are configured to execute the instructions to identify thefirst and second participants among the plurality of users through acomparison process using the face images acquired from the first andsecond participants and the face images registered in the user database.5. A conference assistance system comprising: a first terminal used by afirst participant participating in a conference; a second terminal usedby a second participant participating in the same conference as theconference in which the first participant participates; and a serverdevice, wherein the server device includes one or more memories storinginstructions; and one or more processors configured to execute theinstructions to: acquire a profile of each of a plurality of users whouses the system, and provide a profile related to the second participantto the first terminal.
 6. The conference assistance system according toclaim 5, wherein the one or more processors are configured to executethe instructions to: acquire a face image and the profile of each of theplurality of users, and provide a face image and a profile of the secondparticipant to the first terminal.
 7. The conference assistance systemaccording to claim 6, wherein the one or more processors are configuredto execute the instructions to: generate a user ID for identifying eachof the plurality of users, and acquire the face image and the profile ofthe second participant from a user database that stores the generateduser ID, face image, and profile in association by using a user ID ofthe second participant.
 8. The conference assistance system according toclaim 7, wherein the one or more processors are configured to executethe instructions to identify the first and second participants among theplurality of users through a comparison process using the face imagesacquired from the first and second participants and the face imagesregistered in the user database.
 9. The conference assistance systemaccording to claim 5, wherein the first terminal requests the serverdevice to provide information regarding the profile related to thesecond participant.
 10. The conference assistance system according toclaim 9, wherein the first terminal acquires a response to the requestfor information provision of the profile from the server device anddisplays the profile related to the second participant based on theacquired response.
 11. The conference assistance system according toclaim 10, wherein the one or more processors are configured to executethe instructions to transmit,to the second terminal, speaker informationincluding at least content of a speech by the first participant when thespeech of the first participant is detected, the second terminaltransmits view information including a view of the second participant onthe content of the speech by the first participant to the server device,the server device further includes a participant’s view database thatmanages the content of the speech by the first participant and a view ofthe second participant in association, and the one or more processorsare configured to execute the instructions to transmit a responseincluding statistical information regarding the view of the secondparticipant to the first terminal when a request to provide informationregarding the profile is acquired from the first terminal.
 12. Theconference assistance system according to claim 11, wherein the one ormore processors are configured to execute the instructions to calculatea ratio of speech with which the second participant agrees among speechby the first participant as an agreement ratio and transmit a responseincluding the calculated agreement ratio to the first terminal.
 13. Theconference assistance system according to claim 11, wherein the secondterminal transmits agreement or disagreement of the second participantabout the content of the speeches of the first participant as the viewinformation to the server device.
 14. A conference assistance methodcomprising: in a server device, acquiring a profile of each of aplurality of users using a conference assistance system; and providing afirst participant participating in a conference among the plurality ofusers with a profile related to a second participant participating inthe same conference as the conference in which the first participantparticipates.
 15. (canceled)